Autonomously acting robot that changes pupil image of the autonomously acting robot

ABSTRACT

A monitor is installed in an eye of a robot, and an eye image is displayed on the monitor. The robot extracts a feature quantity of an eye of a user from a filmed image of the user. The feature quantity of the eye of the user is reflected in the eye image. For example, a feature quantity is a size of a pupillary region and a pupil image, and a form of an eyelid image. Also, a blinking frequency or the like may also be reflected as a feature quantity. Familiarity with respect to each user is set, and which user&#39;s feature quantity is to be reflected may be determined in accordance with the familiarity.

RELATED APPLICATIONS

The present application is a continuation of International ApplicationNo. PCT/JP2017/041619, filed Nov. 20, 2017, which claims priority fromJapanese Application No. 2016-227681, filed Nov. 24, 2016, thedisclosures of which applications are hereby incorporated by referenceherein in their entirety.

TECHNICAL FIELD

The present invention relates to a robot that autonomously selects anaction in accordance with an internal state or an external environment.

BACKGROUND ART

A human keeps a pet in a quest for solace. Meanwhile, for variousreasons such as not being able to secure sufficient time to look after apet, not having a living environment in which a pet can be kept, havingan allergy, or hating the thought of being parted by death, there aremany people who give up on keeping a pet. If there were a robot thatperformed the role of a pet, it may be that people who cannot keep a petwould also be provided with the kind of solace that a pet provides(refer to Patent Document 1). Although robot technology has advancedswiftly in recent years, the technology has not advanced so far as torealize a presence as a pet-like companion.

CITATION LIST Patent Literature

-   Patent Document 1: JP-A-2000-323219

Non-Patent Literature

-   Non-patent Document 1: VITOR F. PAMPLONA, MANUEL M. OLIVEIRA and    GLADIMIR V. G. BARANOSKI, “Photorealistic Models for Pupil Light    Reflex and Iridal Pattern Deformation” ACM Transactions on Graphics,    Vol. 28, No. 4, Article 106, August 2009

SUMMARY OF INVENTION Technical Problem

It is commonly said that “a pet looks like its master”. As a human has atrait of feeling attachment to something he or she is accustomed toseeing, it may be that a human subconsciously chooses a pet thatresembles him or her. Also, a pet changes in appearance and facialexpression as it grows in size and increases in age. Of the appearanceand facial expression, eyes are particularly important portions whenrepresenting an expression, individuality, maturity, and the like. Theinventor believes that empathy toward a robot can be greatly increasedby strengthening a power of expression of the robot's eyes.

The invention, having been completed based on the heretofore describedfocus of the inventor, has a main object of providing technology thatincreases a power of expression of an eye of a robot.

Solution to Problem

An autonomously acting robot in an aspect of the invention includes anoperation control unit that selects a motion, a drive mechanism thatexecutes a motion selected by the operation control unit, an eyegenerating unit that generates an eye image, an eye display unit thatcauses the eye image to be displayed in a face region of the robot, anda feature extracting unit that extracts a feature quantity of an eye ofa user from a filmed image of the user.

The eye generating unit causes the feature quantity of the eye of theuser to be reflected in the eye image.

An autonomously acting robot in another aspect of the invention includesan operation control unit that selects a motion, a drive mechanism thatexecutes a motion selected by the operation control unit, an eyegenerating unit that generates an eye image, and an eye display unitthat causes the eye image to be displayed in a face region of the robot.

The eye generating unit causes an aging change of the eye image.

An autonomously acting robot in another aspect of the invention includesan operation control unit that selects a motion, a drive mechanism thatexecutes a motion selected by the operation control unit, an eyegenerating unit that generates an eye image, an eye display unit thatcauses the eye image to be displayed in a face region of the robot, anda light detecting unit that detects an external light source.

The eye generating unit causes a catch light to be included in the eyeimage, and causes a position of the catch light to change in accordancewith a direction of the external light source.

An autonomously acting robot in another aspect of the invention includesan operation control unit that selects a motion, a drive mechanism thatexecutes a motion selected by the operation control unit, an eyegenerating unit that generates an eye image, an eye display unit thatcauses the eye image to be displayed in a face region of the robot, anda light detecting unit that detects an external light.

The eye generating unit causes a size of a pupil image included in theeye image to change in accordance with an intensity of the externallight.

An autonomously acting robot in another aspect of the invention includesa camera, an operation control unit that selects a motion, a drivemechanism that executes a motion selected by the operation control unit,an eye generating unit that generates an eye image, and an eye displayunit that causes the eye image to be displayed in a face region of therobot.

The eye generating unit causes a subject image filmed by the camera tobe superimposed on the eye image.

An autonomously acting robot in another aspect of the invention includesan operation control unit that selects a motion, a drive mechanism thatexecutes a motion selected by the operation control unit, an eyegenerating unit that generates an eye image, an eye display unit thatcauses the eye image to be displayed in a face region of the robot, asight line detecting unit that detects a sight line of a user, and anicon storage unit that stores an icon correlated to a display condition.

When a display condition of any icon is satisfied and a sight linetoward the robot is detected, the eye generating unit superimposes theicon on the eye image.

An autonomously acting robot in another aspect of the invention includesan operation control unit that selects a motion, a drive mechanism thatexecutes a motion selected by the operation control unit, an eyegenerating unit that generates an eye image, an eye display unit thatcauses the eye image to be displayed in a face region of the robot, andan icon storage unit that stores an icon correlated to a displaycondition.

When a display condition of any icon is satisfied, the eye generatingunit superimposes the icon on the eye image at a timing of executing ablinking of the eye image.

Advantageous Effects of Invention

According to the invention, a power of expression of an eye of a robotis easily increased.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A is a front external view of a robot.

FIG. 1B is a side external view of the robot.

FIG. 2 is a sectional view schematically representing a structure of therobot.

FIG. 3 is a configuration diagram of a robot system.

FIG. 4 is a schematic view of an emotion map.

FIG. 5 is a hardware configuration diagram of the robot.

FIG. 6 is a functional block diagram of the robot system.

FIG. 7 is an external view of an eye image.

FIG. 8 is an enlarged view of the eye image.

FIG. 9A is the eye image when an eyelid image is not displayed.

FIG. 9B is the eye image when the eyelid image is drooping on an innereye corner side.

FIG. 9C is a first eye image when the eyelid image is drooping on anouter eye corner side.

FIG. 9D is a second eye image when the eyelid image is drooping on theouter eye corner side.

FIG. 9E is the eye image when a lower eyelid is also displayed inaddition to the eyelid image.

FIG. 10 is an external view of the eye image when displaying areflection.

FIG. 11 is an external view of the eye image when an icon is displayedsuperimposed on a pupillary region.

FIG. 12 is an external view of the eye image when a catch light ischanged to an icon.

DESCRIPTION OF EMBODIMENTS

FIG. 1A is a front external view of a robot 100. FIG. 1B is a sideexternal view of the robot 100.

The robot 100 in this embodiment is an autonomously acting robot thatdetermines an action or gesture based on an external environment and aninternal state. The external environment is recognized using variouskinds of sensor, such as a camera or a thermosensor. The internal stateis quantified as various parameters that express emotions of the robot100. These will be described hereafter.

In principle, the robot 100 has an interior of an owner's home as anaction range. Hereafter, a human involved with the robot 100 will becalled a “user”, and a user forming a member of a home to which therobot 100 belongs will be called an “owner”.

A body 104 of the robot 100 has a rounded form all over, and includes anouter skin formed of a soft material having elasticity, such asurethane, rubber, a resin, or a fiber. The robot 100 may be clothed. Bythe body 104, which is rounded, soft, and pleasant to touch, beingadopted, the robot 100 provides a user with a sense of security and apleasant tactile sensation.

A total weight of the robot 100 is 15 kilograms or less, preferably 10kilograms or less, and more preferably still 5 kilograms or less. Amajority of babies start to walk by themselves by 13 months after birth.An average weight of a baby 13 months after birth is a little over 9kilograms for boys, and a little under 9 kilograms for girls. Because ofthis, when the total weight of the robot 100 is 10 kilograms or less, auser can hold the robot 100 with an effort practically equivalent tothat of holding a baby that cannot walk by itself. An average weight ofa baby less than 2 months after birth is less than 5 kilograms for bothboys and girls. Consequently, when the total weight of the robot 100 is5 kilograms or less, a user can hold the robot 100 with an effortpractically equivalent to that of holding a very young baby.

Advantages of a user holding the robot 100 easily, and wanting to holdthe robot 100, are realized by the attributes of appropriate weight androundness, softness, and pleasantness of touch. For the same reasons, aheight of the robot 100 is desirably 1.2 meters or less, or preferably0.7 meters or less. Being able to be held is an important concept of therobot 100 in this embodiment.

The robot 100 includes three wheels for three-wheeled traveling. Asshown in the drawings, the robot 100 includes a pair of front wheels 102(a left wheel 102 a and a right wheel 102 b) and one rear wheel 103. Thefront wheels 102 are drive wheels, and the rear wheel 103 is a drivenwheel. Although the front wheels 102 have no steering mechanism,rotational speed and a direction of rotation can be individuallycontrolled. The rear wheel 103 is formed of a so-called omni wheel, androtates freely in order to cause the robot 100 to move forward and back,and left and right. By controlling so that the rotational speed of theright wheel 102 b is greater than that of the left wheel 102 a, therobot 100 can turn left or rotate counterclockwise. By controlling sothat the rotational speed of the left wheel 102 a is greater than thatof the right wheel 102 b, the robot 100 can turn right or rotateclockwise.

The front wheels 102 and the rear wheel 103 can be completely stored inthe body 104 using a drive mechanism (a pivoting mechanism and a linkingmechanism). A greater portion of each wheel is hidden by the body 104when traveling too, but when each wheel is completely stored in the body104, the robot 100 is in a state of being unable to move. That is, thebody 104 descends, and sits on a floor surface F, in accompaniment to anoperation of the wheels being housed. In the sitting state, a flatseating face 108 (a ground bottom face) formed in a bottom portion ofthe body 104 comes into contact with the floor surface F.

The robot 100 has two arms 106. The arms 106 do not have a function ofgripping an object. The arms 106 can perform simple actions such asraising, waving, and oscillating. The two arms 106 can also becontrolled individually.

An eye 110 is capable of an image display using a liquid crystal elementor an organic EL element. Various sensors, such as a microphone array oran ultrasonic sensor that can identify a direction of a sound source,are mounted in the robot 100. Also, a speaker is incorporated, and therobot 100 is also capable of simple speech.

A horn 112 is attached to a head portion of the robot 100. As the robot100 is lightweight, as heretofore described, a user can also lift up therobot 100 by grasping the horn 112. An omnidirectional camera isattached to the horn 112, and can film a whole of a region above therobot 100 at one time.

FIG. 2 is a sectional view schematically representing a structure of therobot 100.

As shown in FIG. 2 , the body 104 of the robot 100 includes a base frame308, a main body frame 310, a pair of wheel covers 312 made of resin,and an outer skin 314. The base frame 308 is formed of metal, andsupports an internal mechanism together with configuring a shaft of thebody 104. The base frame 308 is configured by an upper plate 332 and alower plate 334 being linked vertically by a multiple of side plates336. A sufficient interval is provided between the multiple of sideplates 336 so that ventilation is possible. A battery 118, a controldevice 342, and various kinds of actuator are housed inside the baseframe 308.

The main body frame 310 is formed of a resin material, and includes ahead portion frame 316 and a trunk portion frame 318. The head portionframe 316 is of a hollow hemispherical form, and forms a head portionframework of the robot 100. The trunk portion frame 318 is of a steppedcylindrical form, and forms a trunk portion framework of the robot 100.The trunk portion frame 318 is integrally fixed to the base frame 308.The head portion frame 316 is attached to an upper end portion of thetrunk portion frame 318 so as to be relatively displaceable.

Three shafts, those being a yaw shaft 320, a pitch shaft 322, and a rollshaft 324, and an actuator 326 for driving each shaft so as to rotate,are provided in the head portion frame 316. The actuator 326 includes amultiple of servo motors for driving each shaft individually. The yawshaft 320 is driven for a head shaking action, the pitch shaft 322 isdriven for a nodding action, and the roll shaft 324 is driven for a headtilting action.

A plate 325 that supports the yaw shaft 320 is fixed to an upper portionof the head portion frame 316. A multiple of ventilation holes 327 forsecuring ventilation between upper and lower portions are formed in theplate 325.

Abase plate 328 made of metal is provided so as to support the headportion frame 316 and an internal mechanism thereof from below. The baseplate 328 is linked to the plate 325 via a crosslink mechanism 329 (apantagraph mechanism), and is linked to the upper plate 332 (the baseframe 308) via a joint 330.

The trunk portion frame 318 houses the base frame 308 and a wheel drivemechanism 370. The wheel drive mechanism 370 includes a pivot shaft 378and an actuator 379. A lower half portion of the trunk portion frame 318is of a small width in order to form a housing space S of the frontwheel 102 between the wheel covers 312.

The outer skin 314 is formed of urethane rubber, and covers the mainbody frame 310 and the wheel covers 312 from an outer side. The arms 106are molded integrally with the outer skin 314. An aperture portion 390for introducing external air is provided in an upper end portion of theouter skin 314.

FIG. 3 is a configuration diagram of a robot system 300.

The robot system 300 includes the robot 100, a server 200, and amultiple of external sensors 114. The multiple of external sensors 114(external sensors 114 a, 114 b, and so on to 114 n) are installed inadvance in a house. The external sensor 114 may be fixed to a wallsurface of the house, or may be placed on a floor. Positionalcoordinates of the external sensor 114 are registered in the server 200.The positional coordinates are defined as x, y coordinates in the houseenvisaged to be an action range of the robot 100.

The server 200 is installed in the house. The server 200 and the robot100 in this embodiment normally correspond one-to-one. The server 200determines a basic action of the robot 100 based on information obtainedfrom the sensors incorporated in the robot 100 and the multiple ofexternal sensors 114.

The external sensor 114 is for reinforcing sensory organs of the robot100, and the server 200 is for reinforcing brainpower of the robot 100.

The external sensor 114 regularly transmits a wireless signal (hereaftercalled a “robot search signal”) including ID (hereafter called “beaconID”) of the external sensor 114. On receiving the robot search signal,the robot 100 returns a wireless signal (hereafter called a “robotresponse signal”) including beacon ID. The server 200 measures a timefrom the external sensor 114 transmitting the robot search signal untilreceiving the robot response signal, and measures a distance from theexternal sensor 114 to the robot 100. By measuring the distance betweeneach of the multiple of external sensors 114 and the robot 100, theserver 200 identifies the positional coordinates of the robot 100.

Of course, a method whereby the robot 100 regularly transmits its ownpositional coordinates to the server 200 may also be adopted.

FIG. 4 is a schematic view of an emotion map 116.

The emotion map 116 is a data table stored in the server 200. The robot100 selects an action in accordance with the emotion map 116. Theemotion map 116 shown in FIG. 4 shows a magnitude of an emotionalattraction or aversion toward a place of the robot 100. An x axis and ay axis of the emotion map 116 indicate two-dimensional spatialcoordinates. A z axis indicates a magnitude of an emotional attractionor aversion. When a z value is a positive value, an attraction towardthe place is high, and when the z value is a negative value, the robot100 is averse to the place.

On the emotion map 116 of FIG. 4 , a coordinate P1 is a point in anindoor space managed by the server 200 as the action range of the robot100 at which an emotion of attraction is high (hereafter called afavored point). The favored point may be a “safe place”, such as behinda sofa or under a table, or may be a place in which people tend togather or a lively place, like a living room. Also, the safe place maybe a place where the robot 100 was gently stroked or touched in thepast.

A definition of what kind of place the robot 100 favors is arbitrary,but it is generally desirable that a place favored by small children, orby small animals such as dogs or cats, is set as a favored point.

A coordinate P2 is a point at which an emotion of aversion is high(hereafter called a “disliked point”). The disliked point may be a placewhere there is a loud noise, such as near a television, a place wherethere is likely to be a leak, like a bathroom or a washroom, an enclosedspace or a dark place, a place where the robot 100 has been roughlytreated by a user and that invokes an unpleasant memory, or the like.

A definition of what kind of place the robot 100 dislikes is alsoarbitrary, but it is generally desirable that a place feared by smallchildren, or by small animals such as dogs or cats, is set as a dislikedpoint.

A coordinate Q indicates a current position of the robot 100. The server200 identifies positional coordinates of the robot 100, using the robotsearch signal regularly transmitted by the multiple of external sensors114 and the robot response signal responding to the robot search signal.For example, when the external sensor 114 with beacon ID=1 and theexternal sensor 114 with beacon ID=2 each detect the robot 100, theserver 200 obtains the distances of the robot 100 from the two externalsensors 114, and obtains the positional coordinates of the robot 100from the distances.

Alternatively, the external sensor 114 with beacon ID=1 transmits therobot search signal in a multiple of directions, and the robot 100returns the robot response signal when receiving the robot searchsignal. By so doing, the server 200 may ascertain in which direction,and at what distance, the robot 100 is from which external sensor 114.Also, in another embodiment, the server 200 may calculate a distancemoved by the robot 100 from the rotational speed of the front wheel 102or the rear wheel 103, thereby identifying the current position, or mayidentify the current position based on an image obtained from thecamera.

When the emotion map 116 shown in FIG. 4 is provided, the robot 100moves in a direction toward the favored point (coordinate P1), or in adirection away from the disliked point (coordinate P2).

The emotion map 116 changes dynamically. When the robot 100 arrives atthe coordinate P1, the z value (emotion of attraction) at the coordinateP1 decreases with the passing of time. Because of this, the robot 100can emulate animal-like behavior of arriving at the favored point(coordinate P1), “being emotionally satisfied”, and in time “gettingbored” with the place. In the same way, the emotion of aversion at thecoordinate P2 is alleviated with the passing of time. A new favoredpoint or disliked point appears together with the elapse of time,because of which the robot 100 carries out a new action selection. Therobot 100 has “interest” in a new favored point, and ceaselessly carriesout a new action selection.

The emotion map 116 expresses emotional swings as an internal state ofthe robot 100. The robot 100 heads for a favored point, avoids adisliked point, stays for a while at the favored point, and in timeperforms the next action. With this kind of control, the actionselection of the robot 100 can be a human-like or animal-like actionselection.

Maps that affect an action of the robot 100 (hereafter collectivelycalled “action maps”) are not limited to the type of emotion map 116shown in FIG. 4 . For example, various action maps such as curiosity, adesire to avoid fear, a desire to seek safety, and a desire to seekphysical ease such as quietude, low light, coolness, or warmth, can bedefined. Further, an objective point of the robot 100 may be determinedby taking a weighted average of the z values of each of a multiple ofaction maps.

In addition to an action map, the robot 100 has parameters that indicatea magnitude of various emotions or senses. For example, when a value ofa loneliness emotion parameter is increasing, a weighting coefficient ofan action map that evaluates places in which the robot 100 feels at easeis set high, and the value of this emotion parameter is reduced by therobot 100 reaching a target point. In the same way, when a value of aparameter indicating a sense of boredom is increasing, it is sufficientthat a weighting coefficient of an action map that evaluates places inwhich curiosity is satisfied is set high.

FIG. 5 is a hardware configuration diagram of the robot 100.

The robot 100 includes a monitor 170, an internal sensor 128, acommunicator 126, a storage device 124, a processor 122, a drivemechanism 120, and a battery 118. The drive mechanism 120 includes thewheel drive mechanism 370. The processor 122 and the storage device 124are included in the control circuit 342. The units are connected to eachother by a power line 130 and a signal line 132. The battery 118supplies power to each unit via the power line 130. Each unit transmitsand receives a control signal via the signal line 132. The battery 118is a lithium ion rechargeable battery, and is a power source of therobot 100.

The monitor 170 is installed in the eye 110 of the robot 100, and causesan eye image to be displayed (to be described hereafter).

The internal sensor 128 is a collection of various kinds of sensorincorporated in the robot 100. Specifically, the internal sensor 128 isa camera (omnidirectional camera), a microphone array, a distance sensor(infrared sensor), a thermosensor, a touch sensor, an accelerationsensor, a smell sensor, a touch sensor, and the like. The touch sensoris installed between the outer skin 314 and the main body frame 310, anddetects a touch by a user. The smell sensor is an already-known sensorthat applies a principle such that electrical resistance changes inaccordance with an adsorption of molecules that form a source of asmell. The smell sensor classifies various smells into multiple kinds ofcategory.

The communicator 126 is a communication module that carries out wirelesscommunication with the server 200 and various kinds of external device,such as the external sensor 114 and a mobile device possessed by a user,as a target. The storage device 124 is configured of a non-volatilememory and a volatile memory, and stores a computer program and variouskinds of setting information. The processor 122 is means of executing acomputer program. The drive mechanism 120 is an actuator that controlsan internal mechanism. In addition to this, an indicator, a speaker, andthe like are also mounted.

The processor 122 selects an action of the robot 100 while communicatingwith the server 200 or the external sensor 114 via the communicator 126.Various kinds of external information obtained by the internal sensor128 also affect the action selection. The drive mechanism. 120 mainlycontrols the wheel (the front wheel 102) and the head portion (the headportion frame 316). The drive mechanism 120 changes a direction ofmovement and a movement speed of the robot 100 by changing therotational speed and the direction of rotation of each of the two wheels102. Also, the drive mechanism 120 can also raise and lower the wheels(the front wheel 102 and the rear wheel 103). When the wheels rise, thewheels are completely stored in the body 104, and the robot 100 comesinto contact with the floor surface F via the seating face 108, takingon the sitting state. Also, the drive mechanism 120 controls the arm 106via a wire 135.

FIG. 6 is a functional block diagram of a robot system 300.

As heretofore described, the robot system 300 includes the robot 100,the server 200, and the multiple of external sensors 114. Each componentof the robot 100 and the server 200 is realized by hardware including acomputer formed of a CPU (central processing unit), various kinds ofcoprocessor, and the like, a storage device that is a memory or storage,and a wired or wireless communication line that links the computer andthe storage device, and software that is stored in the storage deviceand supplies a processing command to the computer. A computer programmay be configured of a device driver, an operating system, various kindsof application program positioned in an upper layer thereof, and alibrary that provides a common function to the programs. Each blockdescribed hereafter indicates a functional unit block rather than ahardware unit configuration.

One portion of the functions of the robot 100 may be realized by theserver 200, and one portion or all of the functions of the server 200may be realized by the robot 100.

Server 200

The server 200 includes a communication unit 204, a data processing unit202, and a data storage unit 206.

The communication unit 204 manages a process of communicating with theexternal sensor 114 and the robot 100. The data storage unit 206 storesvarious kinds of data. The data processing unit 202 executes variouskinds of process based on data acquired by the communication unit 204and data stored in the data storage unit 206. The data processing unit202 also functions as an interface of the communication unit 204 and thedata storage unit 206.

The data storage unit 206 includes a motion storage unit 232, a mapstorage unit 216, and an individual data storage unit 218.

The robot 100 has a multiple of operation patterns (motions). Variousmotions, such as waving the arm 106, approaching an owner while winding,and watching an owner closely with the head tilted, are defined.

The motion storage unit 232 stores control details of a motion (a motionfile). Each motion is identified by motion ID. The motion file is alsodownloaded into a motion storage unit 160 of the robot 100. Which motionis to be executed may be determined in the server 200, or may bedetermined in the robot 100.

Many motions of the robot 100 are configured as compound motions thatinclude a multiple of unit motions. For example, when the robot 100approaches an owner, the approach may be expressed as a combination of aunit motion of changing direction to face the owner, a unit motion ofapproaching while raising an arm, a unit motion of approaching whileshaking the body, and a unit motion of sitting while raising both arms.By combining these kinds of four motions, a motion of “approaching anowner, raising one arm on the way, and finally sitting after shaking thebody” is realized. An angle of rotation, angular velocity, and the likeof an actuator provided in the robot 100 is defined correlated to a timeaxis in a motion file. Various motions are performed by each actuatorbeing controlled together with the passing of time in accordance withthe motion file (actuator control information).

A shift time for changing from a preceding unit motion to a subsequentunit motion is called an “interval”. It is sufficient that an intervalis defined in accordance with time needed for a unit motion change ordetails of a motion. A length of an interval can be regulated.

Hereafter, settings involved in controlling an action of the robot 100,such as which motion is chosen and when, and output regulation of eachactuator when realizing a motion, will collectively be called“behavioral characteristics”. The behavioral characteristics of therobot 100 are defined by a motion selection algorithm, a motionselection probability, a motion file, and the like.

In addition to a motion file, the motion storage unit 232 stores amotion selection table that defines motions that should be executed whenvarious kinds of event occur. One or more motions, and selectionprobabilities thereof, are correlated to an event in the motionselection table.

In addition to a multiple of action maps, the map storage unit 216 alsostores a map showing a disposition situation of an obstacle such as achair or a table. The individual data storage unit 218 storesinformation on a user, and in particular, on an owner. Specifically, theindividual data storage unit 218 stores master information indicatingfamiliarity toward a user, physical characteristics, and behavioralcharacteristics. The individual data storage unit 218 may also storeattribute information such as age and gender.

The robot 100 has a familiarity internal parameter for each user. Whenthe robot 100 recognizes an action indicating a liking toward the robot100, such as picking the robot 100 up or speaking to the robot 100,familiarity with respect to that user increases. Familiarity decreaseswith respect to a user not involved with the robot 100, a user whobehaves roughly, or a user met infrequently.

The data processing unit 202 includes a position managing unit 208, amap managing unit 210, a recognizing unit 212, an operation control unit222, a familiarity managing unit 220, and a state managing unit 244.

The position managing unit 208 identifies the positional coordinates ofthe robot 100 using the method described using FIG. 3 . The positionmanaging unit 208 may also track positional coordinates of a user inreal time.

The state managing unit 244 manages various kinds of internal parametersuch as a charging rate, an internal temperature, and various kinds ofphysical state such as a processing load of the processor 122. The statemanaging unit 244 includes an emotion managing unit 234.

The emotion managing unit 234 manages various emotion parametersindicating emotions (loneliness, curiosity, a desire for approval, andthe like) of the robot 100. These emotion parameters constantlyfluctuate. The importance of the multiple of action maps changes inaccordance with the emotion parameters, a movement target point of therobot 100 changes in accordance with the action maps, and the emotionparameters change in accordance with movement of the robot 100 and thepassing of time.

For example, when the emotion parameter indicating loneliness is high,the emotion managing unit 234 sets the weighting coefficient of theaction map that evaluates places in which the robot 100 feels at ease tobe high. When the robot 100 reaches a point on the action map at whichloneliness can be eliminated, the emotion managing unit 234 reduces theemotion parameter indicating loneliness. Also, each kind of emotionparameter also changes in accordance with a response action to bedescribed hereafter. For example, the emotion parameter indicatingloneliness decreases when the robot 100 is “hugged” by an owner, and theemotion parameter indicating loneliness increases little by little whenthe robot 100 does not visually recognize an owner for a long time.

The map managing unit 210 changes the parameter of each coordinate onthe multiple of action maps using the method described in connectionwith FIG. 4 . The map managing unit 210 may select one of the multipleof action maps, or may take a weighted average of the z values of themultiple of action maps. For example, it is taken that the z values at acoordinate R1 and a coordinate R2 on an action map A are 4 and 3, andthe z values at the coordinate R1 and the coordinate R2 on an action mapB are −1 and 3. When taking a simple average, the total z value at thecoordinate R1 is 4−1=3, and the total z value at the coordinate R2 is3+3=6, because of which the robot 100 heads in the direction of thecoordinate R2 rather than the coordinate R1.

When the action map A is weighted 5 times with respect to the action mapB, the total z value at the coordinate R1 is 4×5−1=19, and the total zvalue at the coordinate R2 is 3×5+3=18, because of which the robot 100heads in the direction of the coordinate R1.

The recognizing unit 212 recognizes an external environment. Variouskinds of recognition, such as recognition of weather or season based ontemperature and humidity, and recognition of shelter (a safe area) basedon an amount of light and temperature, are included in the recognitionof the external environment. The recognizing unit 156 of the robot 100acquires various kinds of environmental information using the internalsensor 128, and after carrying out a primary processing of theenvironmental information, transfers the environmental information tothe recognizing unit 212 of the server 200.

Specifically, the recognizing unit 156 of the robot 100 extracts animage region corresponding to a moving object, particularly a person oran animal, from an image, and extracts a “feature vector” as acollection of feature quantities indicating physical characteristics andbehavioral characteristics of the moving object from the extracted imageregion. A feature vector component (feature quantity) is a numeralwherein various kinds of physical and behavioral characteristic arequantified. For example, a horizontal width of a human eye is quantifiedin a range of 0 to 1, forming one feature vector component.Already-known facial recognition technology is applied as a method ofextracting a feature vector from a filmed image of a person. The robot100 transmits the feature vector to the server 200.

The recognizing unit 212 of the server 200 further includes a personrecognizing unit 214 and a response recognizing unit 228.

The person recognizing unit 214 determines what person a filmed usercorresponds to by comparing a feature vector extracted from an imagefilmed by the camera incorporated in the robot 100 and a feature vectorof a user (cluster) registered in advance in the individual data storageunit 218 (a user identification process). The person recognizing unit214 includes an expression recognizing unit 230. The expressionrecognizing unit 230 infers an emotion of a user by carrying out imagerecognition of an expression of the user.

The person recognizing unit 214 also carries out a user recognitionprocess on a moving object other than a person, for example, a cat or adog that is a pet.

The response recognizing unit 228 recognizes various responsive actionsperformed with respect to the robot 100, and classifies the actions aspleasant or unpleasant actions. Also, the response recognizing unit 228recognizes a responsive action of an owner with respect to an action ofthe robot 100, thereby classifying the responsive action as a positiveor negative response.

Pleasant and unpleasant actions are distinguished depending on whether aresponsive action of a user is pleasing or unpleasant for an animal. Forexample, being hugged is a pleasant action for the robot 100, and beingkicked is an unpleasant action for the robot 100. Positive and negativeresponses are distinguished depending on whether a responsive action ofa user indicates a pleasant emotion or an unpleasant emotion of theuser. For example, being hugged is a positive response indicating apleasant emotion of the user, and being kicked is a negative responseindicating an unpleasant emotion of the user.

The operation control unit 222 of the server 200 determines a motion ofthe robot 100 in cooperation with an operation control unit 150 of therobot 100. The operation control unit 222 of the server 200 compiles amovement target point of the robot 100, and a movement route for themovement target point, based on an action map selection by the mapmanaging unit 210. The movement control unit 222 compiles a multiple ofmovement routes, and having done so, may select any of the movementroutes.

The operation control unit 222 selects a motion of the robot 100 from amultiple of motions of the motion storage unit 232. A selectionprobability is correlated for each situation to each motion. Forexample, a selection method such that a motion A is executed at aprobability of 20% when a pleasant action is performed by an owner, anda motion B is executed at a probability of 5% when an air temperature is30 degrees or higher, is defined.

A movement target point and a movement route are determined by an actionmap, and a motion is selected in accordance with various kinds of eventto be described hereafter.

The familiarity managing unit 220 manages familiarity for each user. Asheretofore described, familiarity is registered as one portion ofindividual data in the individual data storage unit 218. When a pleasantaction is detected, the familiarity managing unit 220 increasesfamiliarity with respect to that owner. When an unpleasant action isdetected, the familiarity managing unit 220 reduces familiarity. Also,familiarity of an owner not visually recognized for a long periodgradually decreases.

Robot 100

The robot 100 includes a communication unit 142, a data processing unit136, a data storage unit 148, the internal sensor 128, the monitor 170,and the drive mechanism 120.

The communication unit 142 corresponds to the communicator 126 (refer toFIG. 5 ), and manages a process of communicating with the externalsensor 114, the server 200, and another robot 100. The data storage unit148 stores various kinds of data. The data storage unit 148 correspondsto the storage device 124 (refer to FIG. 5 ). The data processing unit136 executes various kinds of process based on data acquired by thecommunication unit 142 and data stored in the data storage unit 148. Thedata processing unit 136 corresponds to the processor 122 and a computerprogram executed by the processor 122. The data processing unit 136 alsofunctions as an interface of the communication unit 142, the internalsensor 128, the drive mechanism 120, and the data storage unit 148.

The data storage unit 148 includes a motion storage unit 160, whichdefines various kinds of motion of the robot 100, and an icon storageunit 172.

Various kinds of motion file are downloaded from the motion storage unit232 of the server 200 into the motion storage unit 160 of the robot 100.A motion is identified by motion ID. An operating timing, an operatingtime, an operating direction, and the like, of the various kinds ofactuator (the drive mechanism 120) are defined chronologically in themotion file in order to perform various motions such as sitting byhousing the front wheels 102, raising the arm 106, causing the robot 100to carry out a rotating action by causing the two front wheels 102 torotate in reverse or by causing only one front wheel 102 to rotate,shaking by causing the front wheels 102 to rotate in a state in whichthe front wheels 102 are housed, or stopping once and looking back whenmoving away from a user.

Various kinds of data may also be downloaded from the map storage unit216 and the individual data storage unit 218 into the data storage unit148.

The icon storage unit 172 stores icon information wherein an icon imageand display conditions thereof are correlated. An icon provides variouskinds of information to a user by being displayed in the eye 110 of therobot, as described hereafter. For example, the communication unit 142of the robot 100 acquires weather information from an external site viathe Internet. When it is raining, or when rain is expected within a fewhours, an icon representing an umbrella is displayed in the eye 110 (tobe described in detail hereafter). In this case, the display conditionsof the umbrella icon are “when it is raining, or when rain is expectedwithin a few hours”. In addition to this, various icons are stored inthe icon storage unit 172. When an air cleanliness sensor is mounted inthe robot 100, an icon may be displayed in accordance with aircleanliness or a chemical substance content. Specifically, when a carbondioxide concentration or a concentration of airborne particulatesreaches a predetermined value or higher, an icon indicating the mattermay be displayed. In addition to this, various icons can be displayed inaccordance with a room temperature, the internal temperature of therobot 100, the charging rate of the battery 118, the existence orotherwise of a failure, and an emotional state managed by the emotionmanaging unit 234.

The internal sensor 128 includes a camera 134, a light detecting unit138, and a sight line detecting unit 140. The camera 134 in thisembodiment is an omnidirectional camera attached to the horn 112. Thelight detecting unit 138 is a light sensor, and detects a direction ofan external light source, a light color, and an amount of light. Thesight line detecting unit 140 is an already-known sensor that detectseye movement of a user from an image filmed by the camera 134. In theembodiment, a gaze of a user toward the robot 100 is detected by thesight line detecting unit 140.

The data processing unit 136 includes the recognizing unit 156, amovement control unit 150, an eye generating unit 152, and an eyedisplay unit 154.

The operation control unit 150 of the robot 100 determines a motion ofthe robot 100 in cooperation with the operation control unit 222 of theserver 200. One portion of motions may be determined by the server 200,and other motions may be determined by the robot 100. Also, aconfiguration may be such that the robot 100 determines a motion, butthe server 200 determines a motion when a processing load of the robot100 is high. A motion that forms a base may be determined by the server200, and an additional motion determined by the robot 100. It issufficient that a way in which a motion determining process is dividedbetween the server 200 and the robot 100 is designed in accordance withthe specifications of the robot system 300.

The operation control unit 150 of the robot 100 determines a directionof movement of the robot 100 together with the operation control unit222 of the server 200. Movement based on an action map may be determinedby the server 200, and an immediate movement such as avoiding anobstacle may be determined by the operation control unit 150 of therobot 100. The drive mechanism 120 causes the robot 100 to head toward amovement target point by driving the front wheel 102 in accordance withan instruction from the operation control unit 150.

The operation control unit 150 of the robot 100 instructs the drivemechanism 120 to execute a selected motion. The drive mechanism 120controls each actuator in accordance with the motion file.

The operation control unit 150 can also execute a motion of holding upboth arms 106 as a gesture asking for “a hug” when a user with a highdegree of familiarity is nearby, and can also perform a motion of nolonger wanting to be hugged by repeatedly causing the left and rightfront wheels 102 to alternately rotate in reverse and stop in a housedstate when bored of the “hug”. The drive mechanism 120 causes the robot100 to perform various motions by driving the front wheel 102, the arm106, and the neck (head port ion frame 316) in accordance with aninstruction from the operation control unit 150.

The eye generating unit 152 generates an eye image. The eye generatingunit 152 individually controls two eye images corresponding to the twoeyes 110. The eye display unit 154 causes an eye image generated by theeye generating unit 152 to be displayed on the monitor 170 installed inthe eye 110. Details of an eye image will be described hereafter.

The recognizing unit 156 of the robot 100 analyzes external informationobtained from the internal sensor 128. The recognizing unit 156 iscapable of visual recognition (a visual unit), smell recognition (anolfactory unit), sound recognition (an aural unit), and tactilerecognition (a tactile unit).

The recognizing unit 156 regularly films an exterior angle using theincorporated omnidirectional camera, and detects a moving object such asa person or a pet. The recognizing unit 156 includes a featureextracting unit 146. The feature extracting unit 146 extracts a featurevector from a filmed image of a moving object. As heretofore described,a feature vector is a collection of parameters (feature quantities)indicating physical characteristics and behavioral characteristics of amoving object. When a moving object is detected, physicalcharacteristics and behavioral characteristics are also extracted fromthe smell sensor, an incorporated highly directional microphone, atemperature sensor, and the like. For example, when a moving objectappears in an image, various characteristics such as having a beard,being active early in the morning, wearing red clothing, smelling ofperfume, having a loud voice, wearing spectacles, wearing a skirt,having white hair, being tall, being plump, being suntanned, and beingon a sofa, are extracted. These characteristics are also quantified,forming feature vector components.

The robot system 300 clusters users appearing with a high frequency as“owners” based on physical characteristics and behavioralcharacteristics obtained from a large amount of image information orother sensing information.

For example, when a moving object (user) having a beard is often activein the early morning (gets up early) and rarely wears red clothing, afirst profile of a cluster (user) that gets up early, has a beard, anddoes not often wear red clothing is created. Meanwhile, when a movingobject wearing spectacles often wears a skirt but the moving object doesnot have a beard, a second profile of a cluster (user) that wearsspectacles and wears a skirt but definitely does not have a beard iscreated.

Although the above is a simple example, the first profile, correspondingto a father, and the second profile, corresponding to a mother, areformed using the heretofore described method, and the robot 100recognizes that there are at least two users (owners) in the house.

Note that the robot 100 does not need to recognize that the firstprofile is the “father”. It is always sufficient that the robot 100 canrecognize a personal profile that is a “cluster that has a beard, oftengets up early, and hardly ever wears red clothing”. A feature vector inwhich the profile is characterized is defined for each profile.

It is assumed that the robot 100 newly recognizes a moving object (user)in a state wherein this kind of cluster analysis is completed.

At this time, the person recognizing unit 214 of the server 200 executesa user identification process based on the feature vector of the newmoving object, and determines what profile (cluster) the moving objectcorresponds to. For example, when a moving object having a beard isdetected, the probability that the moving object is the father is high.When the moving object is active early in the morning, it is even morecertain that the moving object corresponds to the father. Meanwhile,when a moving object wearing spectacles is detected, there is apossibility that the moving object is the mother. When the moving objecthas a beard, the moving object is neither the mother nor the father,because of which the person recognizing unit 214 determines that themoving object is a new person who has not been cluster analyzed.

Formation (cluster analysis) of a cluster (profile) by featureextraction and matching with a cluster in accompaniment to featureextraction may be executed concurrently.

Of a series of recognition processes including detecting, analyzing, anddetermining, the recognizing unit 156 of the robot 100 carries out aselection and extraction of information necessary for recognition, andan analyzing process such as determining is executed by the recognizingunit 212 of the server 200. The recognition processes may be carried outby the recognizing unit 212 of the server 200 alone, or carried out bythe recognizing unit 156 of the robot 100 alone, or the two may executethe recognition processes while dividing roles as heretofore described.

When a strong force is applied to the robot 100, the recognizing unit156 recognizes this using the incorporated acceleration sensor, and theresponse recognizing unit 228 of the server 200 recognizes that a“violent action” has been performed by a user in the vicinity. When auser picks the robot 100 up by grabbing the horn 112, this may also berecognized as a violent action. When a user in a state of facing therobot 100 speaks in a specific volume region and a specific frequencyband, the response recognizing unit 228 of the server 200 may recognizethat a “speaking action” has been performed with respect to the robot100. Also, when a temperature in the region of body temperature isdetected, the response recognizing unit 228 recognizes that a “touchingaction” has been performed by a user, and when upward acceleration isdetected in a state in which touching is recognized, the responserecognizing unit 228 recognizes that a “hug” has been performed.Physical contact when a user picks up the body 104 may also be sensed,and a hug may also be recognized by a load acting on the front wheels102 decreasing.

To summarize, the robot 100 acquires an action of a user as physicalinformation using the internal sensor 128, the response recognizing unit228 of the server 200 determines whether the action is pleasant orunpleasant, and the recognizing unit 212 of the server 200 executes auser identification process based on the feature vector.

The response recognizing unit 228 of the server 200 recognizes variouskinds of response by a user toward the robot 100. “Pleasant” or“unpleasant”, “positive” or “negative” is correlated to one portion oftypical responsive actions among various kinds of responsive action. Ingeneral, almost all responsive actions that are pleasant actions arepositive responses, and almost all responsive actions that areunpleasant actions are negative responses. Pleasant and unpleasantactions relate to familiarity, and positive and negative responsesaffect action selection of the robot 100.

The familiarity managing unit 220 of the server 200 changes thefamiliarity toward a user in accordance with a responsive actionrecognized by the recognizing unit 156. Essentially, the familiaritytoward a user who carries out a pleasant action increases, while thefamiliarity toward a user who carries out an unpleasant actiondecreases.

The recognizing unit 212 of the server 200 may determine whether aresponse is pleasant or unpleasant, and the map managing unit 210 of theserver 200 may change the z value of the point at which the pleasant orunpleasant action has been carried out on an action map that represents“attachment to a place”. For example, when a pleasant action is carriedout in a living room, the map managing unit 210 may set a favored pointat a high probability in the living room. In this case, a positivefeedback advantage is realized in that the robot 100 favors the livingroom, and further favors the living room due to being the recipient of apleasant action in the living room.

Depending on what kind of action the robot 100 is subjected to by amoving object (user), familiarity with respect to the user changes.

The robot 100 sets a high familiarity for a frequently met person, aperson who frequently touches the robot 100, and a person who frequentlyspeaks to the robot 100. Meanwhile, familiarity decreases for a rarelyseen person, a person who does not often touch the robot 100, a violentperson, and a person who scolds in a loud voice. The robot 100 changesthe familiarity of each user based on various items of exterior angleinformation detected by the sensors (visual, tactile, and aural).

The actual robot 100 autonomously carries out a complex action selectionin accordance with an action map. The robot 100 acts while beingaffected by a multiple of action maps based on various parameters suchas loneliness, boredom, and curiosity. When the effect of the actionmaps is removed, or when in an internal state in which the effect of theaction maps is small, the robot 100 essentially attempts to approach aperson with high familiarity, and attempts to move away from a personwith low familiarity.

Actions of the robot 100 are classified below in accordance withfamiliarity.

(1) A user with extremely high familiarity

The robot 100 strongly expresses a feeling of affection by approaching auser (hereafter called an approaching action), and performing anaffectionate gesture defined in advance as a gesture indicating goodwilltoward a person.

(2) A user with comparatively high familiarity

The robot 100 carries out only an approaching action.

(3) A user with comparatively low familiarity

The robot 100 does not carry out any special action.

(4) A user with particularly low familiarity

The robot 100 carries out a withdrawing action.

According to the heretofore described control method, the robot 100approaches the user when finding a user with high familiarity, andconversely, moves away from the user when finding a user with lowfamiliarity. According to this kind of control method, the robot 100 canexpress by behavior a so-called “shyness”. Also, when a visitor (a userA with low familiarity) appears, the robot 100 may move away from thevisitor and head toward a family member (a user B with highfamiliarity). In this case, user B can perceive that the robot 100 isshy and feeling uneasy, and relying on user B. Owing to this kind ofbehavioral expression, pleasure at being chosen and relied upon, and anaccompanying feeling of affection, are evoked in user B.

Meanwhile, when user A, who is a visitor, visits frequently, and speaksto and touches the robot 100, familiarity of the robot 100 toward user Agradually rises, and the robot 100 ceases to perform an action ofshyness (a withdrawing action) with respect to user A. User A can alsofeel affection toward the robot 100 by perceiving that the robot 100 hasbecome accustomed to user A.

The heretofore described action selection need not necessarily beexecuted constantly. For example, when an internal parameter indicatingcuriosity of the robot 100 is high, weight is given to an action mapfrom which a place in which the curiosity is satisfied is obtained,because of which there is also a possibility that the robot 100 does notselect an action affected by familiarity. Also, when the external sensor114 installed in the hall detects the return home of a user, the robot100 may execute an action of greeting the user with maximum priority.

FIG. 7 is an external view of an eye image 174.

The eye generating unit 152 generates the eye image 174, which includesa pupil image 164 and a peripheral edge image 168. The eye generatingunit 152 causes the eye image 174 to be displayed as a moving image.Specifically, a sight line of the robot 100 is represented by moving thepupil image 164. Also, a blinking action is executed at a predeterminedtiming. The eye generating unit 152 performs various movements of theeye image 174 in accordance with various operation patterns. The eyedisplay unit 154 causes the eye image 174 to be displayed on the monitor170 of the eye 110. The monitor 170 desirably has a curved form, in thesame way as a human eyeball. The monitor 170 in the embodiment isdescribed as being of a flat form.

The pupil image 164 includes a pupillary region 158 and a corneal region162. Also, a catch light 166 for representing a reflection of externallight is also displayed in the pupil image 164. Rather than shiningowing to a reflection of external light, the catch light 166 of the eyeimage 174 is an image region represented as a region of high luminanceby the eye generating unit 152.

The eye generating unit 152 can cause the pupil image 164 to movevertically and horizontally on the monitor 170. When the recognizingunit 156 of the robot 100 recognizes a moving object, the eye generatingunit 152 generates an operation pattern (moving image data) such thatthe pupil image 164 is directed in the direction in which the movingobject exists. The eye display unit 154 represents the “sight line” ofthe robot 100 by causing the display of the eye image 174 to change inaccordance with the operation pattern. As heretofore described, the eyegenerating unit 152 individually controls eye images 174 displayed inthe two eyes 110. When the sight line of the robot 100 is directedtoward a target such as a user, the eye generating unit 152 directs bothof the two eye images 174 toward the target.

The eye generating unit 152 may cause a form of the pupil image 164 tochange in accordance with a position on the monitor 170. Specifically, aperfectly circular form is adopted when the pupil image 164 is in thecenter of the monitor 170, and the form is changed to an elliptical formwhen the pupil image 164 is in a peripheral edge portion. By the form ofthe pupil image 164 being changed in accordance with the position, themonitor 170 can be caused to appear to be of a curved form like anactual eyeball, even in the case of the flat monitor 170. The eyegenerating unit 152 represents the eye image 174 as a planar image bygenerating a three-dimensional object resembling an eyeball, andprojecting the three-dimensional object onto the monitor 170 (projectionsurface).

The light detecting unit 138 detects an external light source such asthe sun or an electric light. The eye generating unit 152 causes aposition of the catch light 166 to change in response to a direction inwhich the external light source exists. FIG. 7 shows a display positionof the catch light 166 when an external light source exists above and tothe left as seen from the robot 100. By linking the position of thecatch light 166 to an external light source, a more realistic eye image174 can be represented. Also, distorting the catch light 166 into anelliptical form in accordance with the position of the catch light 166,in the same way as the pupil image 164, is effective in causing themonitor 170 to appear to be of a curved form.

FIG. 8 is an enlarged view of the eye image 174.

The eye image 174 in the embodiment also includes an eyelid image 176showing an eyelid superimposed on the pupil image 164 and the peripheraledge image 168. The eyelid image 176 includes an eyelash 178. Theperipheral edge image 168 is a portion corresponding to a humanconjunctiva. The corneal region 162 included in the pupil image 164 is aportion corresponding to a human cornea. An iris may be displayed in thecorneal region 162.

Of the eye image 174, the eye generating unit 152 causes the eyelidimage 176, the pupillary region 158, the corneal region 162, and thecatch light 166 to change. There are two kinds of change of the eyeimage 174, those being a constant, stable change (hereafter called a“constant change”) and a temporary change (hereafter called a “transientchange”).

Furthermore, there are two kinds of constant change, those being anirreversible change accompanying a passing of time (hereafter called an“aging change”) and a change that incorporates features of an owner'seye (hereafter called a “reflective change”). That is, the eye 110 ofthe robot 100 has a feature of gradually coming to resemble an owner'seye.

(1) Constant Change

(1-1) Aging Change

It is said that the diameter of a human pupil decreases with aging. Inaccompaniment to an increase in age, the whiteness of a conjunctivafades, and the muscle strength that closes an eyelid diminishes, becauseof which the eyelid gradually droops. Also, it is said that thefrequency of blinking in infancy is low, and the frequency of blinkingstabilizes after reaching adulthood.

The eye generating unit 152 represents change accompanying growth andaging of an eye of a living being, particularly a human, in the eyeimage 174. The eye generating unit 152 causes the eye image 174 tochange gradually in accompaniment to the passing of time from the timethe robot 100 is shipped from the factory. Specifically, the eyegenerating unit 152 reduces a diameter of the corneal region 162 inaccompaniment to the passing of time, and causes a fixed position of theeyelid image 176 to descend. After a particularly long time elapses, asetting may be such that the eyelid image 176 does not rise above apredetermined position. Although blinking is represented by the eyegenerating unit 152 regularly lowering the eyelid image 176, thefrequency of blinking also changes in accompaniment to the aging of therobot 100.

(1-2) Reflective Change

An eye is an important region that is liable to leave an impression onanother person. The feature extracting unit 146 films an eye of anowner, and extracts features of the eye. Feature extraction is such thatfeatures of an external appearance of the eye are extracted as featurequantities by the features being quantified. Not only static featuressuch as a pupil size, but also dynamic features such as a sight linemovement habit or blinking frequency, may be included in the externalappearance of the eye. More specifically, parameters such as eyelid formand fixed position, how the eyelid descends, eyelash length, and pupilsize and color, are extracted as feature quantities of the eye. Dynamicfeatures such as a fixed position of the pupil, a habit of glancingupward or narrowing the eyes, or blinking frequency, also form featurequantities. In the case of a female, eyelash length and the color of theeyelid image 176 may change depending on makeup.

The eye generating unit 152 causes the eye 110 of the robot 100 toresemble an owner by causing the feature quantities of the owner's eyeto also be reflected in the eye image 174. In the embodiment, the eyegenerating unit 152 spends several months to several years causing thefeatures of the owner to be reflected in the eye image 174. For example,when the robot 100 belongs to an owner whose eyelid droops somewhat onan outer eye corner side, the eye generating unit 152 causes the eyelidimage 176 of the robot 100 to change so that the outer eye corner sidedroops.

The eye generating unit 152 causes the eye image 174 to change byadjusting a spline curve that forms a boundary line between variouskinds of portion such as the eyelid image 176 and the pupil image 164,or more specifically, by adjusting one or more control points on aspline curve. For example, the eye generating unit 152 causes the formof the eyelid image 176 to change by changing the spline curve formingthe boundary between the eyelid image 176 and the pupil image 164. Bygradually causing the spline curve control points to change, the look ofthe eye image 174 can be caused to change little by little over time.

The eye generating unit 152 may prepare image patterns of a multiple ofthe eye image 174. Further, the look of the eye image 174 may be causedto change by switching among the multiple of eye images 174.

When there are a multiple of owners, the eye generating unit 152 causesthe features of the owner with the highest familiarity to be reflectedin the eye image 174. When the owner with the highest familiaritychanges from an owner P1 to an owner P2, the eye generating unit 152selects the features of the owner P2 as a new reflective change model.

Owing to the aging change and the reflective change, the robot 100gradually grows and ages, and comes to resemble an owner. Because ofthis, the owner can feel a sense of closeness to the robot 100, and cansense the growth and aging of the robot 100. The eye 100 may be causedto constantly change based on only one of the aging change or thereflective change.

(2) Transient Change

A transient change mainly occurs due to external light. Althoughtransient changes due to a display of a “reflection” or an “icon” alsooccur in the eye image 174, these will be described hereafter. Asheretofore described, the eye generating unit 152 causes the position ofthe catch light 166 to change in response to the direction of anexternal light. Also, the eye generating unit 152 causes the form, acolor, and a size of the catch light 166 to change in accordance with aform and an intensity of an external light source. For example, when theexternal light source is a window, the eye generating unit 152 causesthe catch light 166 to change to the form of the window. When theexternal light source is blue, the catch light 166 becomes blue. Thehigher the light intensity, the further the eye generating unit 152increases the size of the catch light 166. When there are a multiple oflight sources, the eye generating unit 152 causes a multiple of catchlights 166 to be displayed.

When a sight line from a user toward the robot 100 is detected, the eyegenerating unit 152 may increase the size of the catch light 166. Thesight line detecting unit 140 determines whether or not a user is gazingat the robot 100 by analyzing a filmed image and detecting a pupillaryorientation of the user. Also, a configuration may be such that sightline detection is executed when a size of a face region of the user inthe filmed image is of a predetermined value or greater, or on conditionthat the user is within a predetermined distance according to thedistance sensor (not shown).

When the light intensity is high, the eye generating unit 152 increasesa diameter of the pupillary region 158. It is sufficient that arelationship between light intensity and pupillary diameter iscontrolled in accordance with an already-known model (for example, referto Expression (2) of Non-patent Document 1). When the light intensity isparticularly high, an “appearance of seeming dazzled” may be representedby lowering the eyelid image 176.

FIG. 9A to FIG. 9E are external views of various eye images 174. Eachdrawing corresponds to a right eye.

FIG. 9A is the eye image 174 when the eyelid image 176 is not displayed.FIG. 9B is the eye image 174 when the eyelid image 176 is drooping on aninner eye corner side. FIG. 9C is the eye image 174 when the eyelidimage 176 is drooping on the outer eye corner side. FIG. 9C is aso-called “drooping eye”. In FIG. 9D too, the eyelid image 176 isdrooping on the outer eye corner side, though to a lesser extent than inFIG. 9C. Also, in FIG. 9E, a lower eyelid 180 is displayed in additionto the eyelid image 176. A portion resembling a bag below the eye canalso be represented by the image of the lower eyelid 180.

By causing features of an owner to be reflected in the size and color ofthe pupil image 164, the angle and form of the eyelid image 176, and thelike, a face resembling a user can be represented. For example, when anowner has eyes that slant downward, the eye generating unit 152 lowersthe fixed position of the eyelid image 176, as in FIG. 9C and FIG. 9D.When an owner has double eyelids, the eye generating unit 152 changesthe eyelid image 176 to a double eyelid image. Features of an owner arealso reflected in the length and number of the eyelash 178 of the eyelidimage 176.

FIG. 10 is an external view of the eye image 174 when displaying areflection.

The eye generating unit 152 causes a subject 182 filmed by the camera134 to be displayed superimposed on the pupil image 164. By so doing, anaspect wherein an object the robot 100 is looking at is reflected in thepupil image 164, particularly the pupillary region 158, is represented.By actively causing the subject 182 to be displayed in the pupil image164, the appearance of the eye 100 can be improved. The eye generatingunit 152 may cause the subject 182 to be displayed distorted in thepupil image 164 as though seen with a fisheye lens. When adopting thiskind of distorted display, the eye image 174 can be caused to appear tobe of a curved form, in the same way as an eyeball.

FIG. 11 is an external view of the eye image 174 when an icon 184 isdisplayed superimposed on the pupillary region 158. FIG. 12 is anexternal view of the eye image 174 when the catch light 166 is changedto the icon 184.

When a predetermined display condition is satisfied, the eye generatingunit 152 causes the icon 184 to be displayed in the pupillary region 158or the catch light 166. For example, when it is sunny, the icon 184indicating sunshine is displayed superimposed on the pupillary region158 (refer to FIG. 11 ). By looking into the pupillary region 158, auser can confirm that it is sunny outside. The icon 184 may indicatefuture weather (a weather forecast) rather than current weather.

The icon 184 may be displayed in the pupillary region 158, or the catchlight 166 may be caused to change to the icon 184. In FIG. 12 , thecatch light 166 is changed to the icon 184 indicating rain.

Provided that a display condition of any icon 184 is satisfied when asight line toward the robot 100 is detected by the sight line detectingunit 140, the eye generating unit 152 causes that icon 184 to bedisplayed. A configuration may be such that the eye generating unit 152causes the icon 184 to be displayed when a sight line is detected for apredetermined time or longer, that is, when a stare is detected.

The sight line detecting unit 140 may determine that a user is lookingat the robot 100 when both eye regions can be detected in a face imageof a size of a certain value or greater. When the state of lookingcontinues for a predetermined time, it is determined that there is a“stare”, and the eye generating unit 152 may display the icon 184 so asto be faint, dull, or small. Further, when the state of lookingcontinues longer, the eye generating unit 152 may display the icon 184so as to become gradually darker, brighter, or larger.

A voice of a user is recognized by a voice recognizing unit (not shown),and the eye generating unit 152 may cause the icon 184 responding to thevoice to be displayed. For example, when asked “how's the weather?” by auser, the eye generating unit 152 may cause the icon 184 indicating thecurrent weather to be displayed in response to the keyword “weather”.When asked “what will the weather be like tomorrow?” by a user, the eyegenerating unit 152 may cause the icon 184 indicating the followingday's weather to be displayed in response to the keywords “tomorrow” and“weather”. When asked “aren't you hungry?” by a user, the eye generatingunit 152, recognizing the meaning of “hungry”, may cause an indicatorimage indicating the charging rate of the battery 118 to be displayed asthe icon 184. A configuration may be such that the eye generating unit152 causes the relevant icon 184 to be displayed on condition that oneof various kinds of utterance is made when a stare has continued for apredetermined time.

The eye generating unit 152 deletes the icon 184 when a predeterminedtime elapses. Alternatively, the eye generating unit 152 may delete theicon 184 on condition that a stare from a user is no longer detected.The icon 184 may be deleted over time using a fade-out method.

The eye generating unit 152 may cause the icon 184 to be displayed at anarbitrary timing, randomly for example, or may always cause the icon 184to be displayed when a display condition is satisfied. Also, the eyegenerating unit 152 may regularly execute a “blinking display” thatcloses the eyelid image 176, and cause the icon 184 to be displayed inaccordance with the blinking timing.

Heretofore, the robot 100 and the robot system 300 including the robot100 have been described based on an embodiment.

According to the embodiment, the eye 110, which expresses individualitywith regard to the external appearance of the robot 100, can be causedto resemble an owner's eye. As the eye 110 of the robot 100 comes toresemble his or her own eye, the owner is liable to feel a sense ofcloseness to the robot 100. Also, of a multiple of owners, the eye 110of the robot 100 comes to resemble an owner with high familiarity,because of which the quantitative value familiarity can be qualitativelysuggested using the eye image 174. Because of this, a desire to be ongood terms with, and be liked by, the robot 100 is aroused in the owner.

Also, as the eye 110 changes with age, growth and aging of the robot 100can be represented using the eye 110. Because of this, an owner isliable to maintain affection toward the robot 100 with which the ownerspends a long time.

When filming a person, there is a technique whereby a catch light iscaused to be reflected in an eye using a stroboscope or the like. Acatch light has an effect of causing a human eye to appear vivacious.The robot 100 in the embodiment is such that the eye generating unit 152actively causes an image of the catch light 166 to be displayed as oneportion of the eye image 174. Originally, a catch light is a reflectionof external light (reflected light), but in the case of the robot 100,cuteness of the eye 110 of the robot 100 is emphasized by causing thecatch light 166 to be displayed.

The eye generating unit 152 causes the position of the catch light 166to change in accordance with the direction of an external light source,because of which a realistic image with no incongruity can berepresented. The eye generating unit 152 causes the size of thepupillary region 158 to change in accordance with light intensity,whereby the eye 110 like that of a living being can be represented. Byrepresenting a reflection by displaying the subject 182 superimposed onthe eye image 174, the appearance of the eye 110 of the robot 100 isimproved. Controlling the blinking frequency and timing also contributesto the sense of realism of the eye 110.

The robot 100 in the embodiment has non-verbal communication as a basicconcept. Meanwhile, the robot 100 can also be caused to function as aninformation terminal. According to the embodiment, various items ofuseful information, such as the weather and the charging rate, can becommunicated as the icon 184 using the eye image 174. The robot 100 alsofunctions as an information terminal of use to a user, while adhering toa basic concept of being non-verbal and sensitive.

It is thought that by configuring so that the icon 184 is displayed asthough floating up only when a user looks closely at the robot 100, theuser will become keen to look closely at the robot 100. When an ownerlooks closely at the robot 100 at close range, the camera 134 easilyextracts a feature vector of the owner. Increasing opportunities for anowner to look closely at the robot 100 is also effective in increasingaccuracy of the user identification process.

The invention not being limited to the heretofore described embodimentor a modified example, components can be changed or embodied withoutdeparting from the scope of the invention. Various inventions may beformed by a multiple of the components disclosed in the heretoforedescribed embodiment or the modified example being combined asappropriate. Also, some components may be eliminated from the total ofcomponents shown in the heretofore described embodiment or the modifiedexample.

Although a description has been given assuming that the robot system 300is configured of one robot 100, one server 200, and the multiple ofexternal sensors 114, one portion of the functions of the robot 100 maybe realized by the server 200, and one portion or all of the functionsof the server 200 may be allocated to the robot 100. One server 200 maycontrol a multiple of the robot 100, or a multiple of the server 200 maycontrol one or more of the robot 100 in cooperation.

A third device other than the robot 100 and the server 200 may manageone portion of functions. A collection of the functions of the robot 100and the functions of the server 200 described in FIG. 7 can also becomprehensively grasped as one “robot”. It is sufficient that a methodof distributing the multiple of functions needed in order to realize theinvention with respect to one or multiple items of hardware isdetermined with consideration to the processing capability of each itemof hardware, specifications required of the robot system 300, and thelike.

As heretofore described, “the robot in a narrow sense” is the robot 100excluding the server 200, but “the robot in a wide sense” is the robotsystem 300. It is thought that there is a possibility of many functionsof the server 200 being integrated in the robot 100 in future.

The eye generating unit 152 may control the eyelid image 176 inaccordance with a time band or the internal state of the robot 100. Forexample, the eyelid image 176 may descend when the charging ratedecreases, and the eyelid image 176 may descend completely, closing theeye, during charging. When it is late at night, “sleepiness” may beexpressed by lowering the eyelid image 176.

The eyelid image 176 in the embodiment is formed as one portion of theeye image 174. As a modified example, the eyelid image 176 and the lowereyelid 180 may be formed as physical covers covering the monitor 170.

When the parameter indicating loneliness reaches a predeterminedthreshold or greater, features of the mother may be reflected, and whenthe parameter indicating curiosity reaches a predetermined threshold orgreater, features of the father may be reflected. Feature quantities ofan owner's eye may be acquired divided by situation. For example, thefeature extracting unit 146 may separately acquire a feature quantity ofthe eye when the owner P1 is laughing and a feature quantity of the eyewhen the owner P1 is sad.

An amount of change (a difference) between the feature quantity of theeye when laughing and a feature quantity of the eye at a normal time istaken to be a feature quantity change A, and an amount of change (adifference) between the feature quantity of the eye when sad and thefeature quantity of the eye at a normal time is taken to be a featurequantity change B. Also, the feature quantity change A of the owner P1is assumed to be greater than the feature quantity change A of the ownerP2. Meanwhile, the feature quantity change B of the owner P2 is assumedto be greater than the feature quantity change B of the owner P1. Inthis case, the eye generating unit 152 may cause features of the ownerP2 to be reflected when the parameter indicating loneliness reaches thepredetermined threshold or greater, and may cause features of the ownerP1 to be reflected when a parameter indicating pleasure reaches apredetermined threshold or greater. When in an emotional state ofhappiness, the robot 100 resembles the owner P1, whose smiling face ischaracteristic. Also, when in an emotional state of loneliness, therobot 100 resembles the owner P2, whose sad face is characteristic.According to this kind of control method, a characteristic expression ofan owner can be synchronized with an emotional state of the robot 100.

The eye generating unit 152 may select an owner that is to be a targetof reflective change in accordance with various kinds of internal statesuch as the charging rate, the processing load of the processor 122, orthe internal temperature of the robot 100, or an external state such aswhen light intensity is of a predetermined value or greater, when it is10 o'clock at night or later, or the weather. According to this kind ofcontrol method, an owner can be caused to feel that the robot 100somehow resembles him or herself only when in a specific state. Also, itis thought that as features of each of a multiple of owners can beincorporated in the robot 100, a large number of owners are liable tofeel a sense of closeness to the robot 100.

When there are a multiple of owners, the eye generating unit 152 maycause features of the multiple of owners to be reflected in the eye 110by taking a weighted average of feature vector components of each ownerbased on the familiarity with respect to each owner.

It is often the case that a human's eyes differ in form between left andright. In the case of the robot 100 too, features of an owner's righteye may be reflected in the right eye 110, and features of an owner'sleft eye may be reflected in the left eye 110.

The eye generating unit 152 may express bleariness of the eye 110 byshaking or blurring the catch light 166 or the pupillary region 158.

Three kinds of process are included in a reflective change, those beinga first process of selecting an owner that is to be a target of thereflective change, a second process of extracting features of theselected owner, and a third process of causing the features of theselected owner to be reflected in the eye image 174. The second processis executed after the first process is completed, and the third processis executed over time after the second process is completed.

The eye generating unit 152 may start a reflective change of the eyeimage 174 immediately after the robot 100 is introduced into the home.For example, the eye generating unit 152 may select the first ownerrecognized after the robot 100 is introduced into the home and the powersupply is turned on as the target of the reflective change (firstprocess). Alternatively, a predetermined grace period until the eyegenerating unit 152 starts a reflective change may be set. The graceperiod is one kind of observation period secured for the first processof the robot 100 selecting an owner that is to be a target of areflective change. After the grace period elapses, the eye generatingunit 152 may start a reflective change with any owner as the target. Forexample, when familiarity with respect to any owner reaches apredetermined threshold or greater, the eye generating unit 152 mayadopt that owner as the target of the reflective change. In this case, aperiod until the familiarity exceeds the threshold is the grace period.The grace period may also be a constant time.

The eye generating unit 152 may record the date and time of the powersupply first being turned on after the robot 100 is introduced into thehome. Alternatively, a timer may be caused to start when the powersupply is first turned on. Not being limited to the power supply beingturned on, it is sufficient that the date and time at which apredetermined trigger event occurs is recorded as a standard date andtime. Based on this kind of time information, the eye generating unit152 may cause a temporal change in the eye image 174.

A multiple of display layers may be included in the eye 110 of the robot100. Specifically, a first layer (a display surface of the eye image174) on which a three-dimensional object representing an eyeball isprojected, a second layer on which the icon 184 is displayed, and athird layer on which the eyelid image 176 is displayed are stacked, anda fourth layer on which special information is displayed may be formedin an uppermost layer (external side). Special information in this caseis information of which there is a high necessity to notify a user, suchas a remaining battery charge being at a predetermined value or lower,or a system error occurring, or information of which notificationrequires urgency, and is defined by a designer of the robot 100. Theicon 184 may be displayed on the first layer (deepest layer) rather thanthe second layer by displaying the icon 184 on the pupil image 164 ofthe three-dimensional object.

Multiple kinds of “temperament” may be set in advance in the robot 100.As an aging change, the eye generating unit 152 of the robot 100 with atemperament A may cause the eyelid image 176 to gradually change to adrooping eye, and the eye generating unit 152 of the robot 100 with atemperament B may cause the eyelid image 176 to gradually change to aslanting eye. In this way, an aging change pattern may be determinedbased on a congenital setting of the robot 100.

Also, the eye image 174 may be caused to undergo an aging change inaccordance with an environment or experience of the robot 100. Forexample, an aging change pattern of the eye image 174 may be determinedin accordance with whether the robot 100 has experienced more pleasantactions or unpleasant actions. The eye generating unit 152 may selectone of multiple kinds of aging change pattern in accordance withconditions such as an owner with familiarity of a first threshold orgreater being present, a multiple of owners with familiarity of a secondthreshold or greater being present, and no owner with familiarity of athird threshold or less being present. As one example, the eyegenerating unit 152 of the robot 100 that has experienced more pleasantactions than unpleasant actions causes the eyelid image 176 to change toa drooping eye, and causes the pupillary region 158 to change to belarger than an initial setting. Meanwhile, the eye generating unit 152of the robot 100 that has experienced more unpleasant actions thanpleasant actions may cause the eyelid image 176 to change to a slantingeye, and cause the pupillary region 158 to change to be smaller than theinitial setting.

The eye generating unit 152 may cause the emotional state of the robot100 to be reflected in the eye image 174. For example, when the robot100 is in a certain emotional state (for example, surprised), the eyegenerating unit 152 may represent an aspect of “opening the eyes wide”by enlarging the pupillary region 158, and when the robot 100 is in adifferent emotional state (for example, agitated), the eye generatingunit 152 may increase the blinking frequency. The eye generating unit152 may cause the eye image 174 to change when blinking, in the same wayas the icon 184. According to this kind of control method, an owner canperceive an emotion of the robot 100 from the eye image 174 of the robot100. In this way, the eye generating unit 152 not only causes an agingchange or a reflective change of the eye image 174, but may also cause atransient change of the eye image 174 in accordance with an emotionalstate.

The eye generating unit 152 may cause the eye image 174 to change inaccordance with a location of the robot 100. For example, when the robot100 exists at a favored point on the emotion map 116, the eye generatingunit 152 may represent an aspect of “opening the eyes wide” by enlargingthe pupillary region 158, and when the robot 100 exists at a dislikedpoint, the eye generating unit 152 may represent an aspect of “beingrestless” by raising the blinking frequency. In this way, an emotion ofthe robot 100 with respect to a place may be expressed using the eyeimage 174.

When displaying a reflection too, the eye generating unit 152 may changethe display of the subject 182 to be darker, larger, and more distinctthe longer a staring state continues.

The eye generating unit 152 may project a filmed image unchanged ontothe pupil image 164, or may project the filmed image after carrying outa processing thereon. For example, when a focal point of a filmed imageis blurred, the eye generating unit 152 may generate an image of asharper subject 182 after performing an already-known image correctionsuch as high-pass filtering.

The eye generating unit 152 may save a multiple of filmed images inadvance in connection with one user, and when an appropriate filmedimage cannot be obtained for a reason such as the user being too close,the eye generating unit 152 may display a saved filmed image as areflection in the pupil image 164 instead of an actual filmed image.According to this kind of processing method, an appropriate image of thesubject 182 can be displayed as a reflection, even when an image filmedby the camera 134 is out of focus. Also, when displaying a reflection bysubstituting with a saved filmed image, the filmed image may be enlargedor reduced in accordance with the distance between the robot 100 and theuser. When the robot 100 and the user are particularly close, the usermay be filmed using a close-up camera.

When a predetermined reset button installed in the robot 100 is pressed,or when a predetermined reset command is transmitted from the server 200to the robot 100, the eye generating unit 152 may reset a constantchange. When a reset is executed, the eye generating unit 152 clears anaging change or a reflective change, thereby changing the eye image 174to the eye image 174 at the time of shipping from the factory. Byresetting, the eye image 174 may be returned to, for example, the eyeimage 174 of a predetermined point several days before or several weeksbefore. In this case, the eye generating unit 152 needs to savechronological information of the constant change of the eye image 174.According to this kind of reset function, the eye image 174 can be“rewound” to the eye image 174 of several days before or several weeksbefore, even when the eye image 174 does not make a preferred constantchange. A reset may be executed by an owner at an arbitrary timing, aquantity limit may be provided, or a reset authority may be limited by apassword or the like.

After resetting too, there is a possibility of the eye image 174 of therobot 100 making the same constant change as the previous time when theexternal environment of the robot 100 does not change. As a modifiedexample, a user may be able to set a change tendency of the eye image174. For example, when a “dignified” change tendency is selected, theeye image 174 may be such that a lowering of the eyelid image 176 isrestricted. When a “cute” change tendency is selected, the eye image 174may be such that the eyelash 178 is set to be longer than normal, andthe diameter of the pupil image 164 is increased in comparison with anormal time. A user may instruct using a voice, or may instruct via anoperating unit installed in the robot 100 or the server 200.

A speed of a constant change (aging change or reflective change) may bevariable. For example, the operation control unit 150 may increase thespeed of an aging change for a period shortly after the purchase of therobot 100, and reduce the speed of the aging change after apredetermined period elapses. According to this kind of control method,growth and aging resembling that of a living being can be expressed inthat the eye image 174 is liable to change when the robot 100 is“young”, and the eye image 174 stabilizes when the robot 100 becomes“mature”.

The robot system 300 does not need to include a function of causing aneye image to change from the time of shipping from the factory.Functional strengthening of the robot system 300 after the robot system300 is shipped may be realized by downloading an eye image controlprogram that realizes an eye image control function via a communicationnetwork.

The following combinations of the features are encompassed by one ormore embodiments of the subject matter described in the presentspecification.

A robot, comprising:

a non-transitory computer readable medium configured to storeinstructions thereon;

a display in a face region of the robot;

a sensor configured to capture an image of a user; and

a processor connected to the non-transitory computer readable medium,wherein the processor is configured to execute the instructions for:

-   -   extracting a feature quantity of an eye of the user from the        captured image;    -   managing an internal state of the robot;    -   generating an eye image, wherein generating the eye image        comprises determining whether to reflect the feature quantity of        in the eye image based on the internal state;    -   instructing the display to display the eye image.

The robot according to the above combinations, wherein the internalstate comprises familiarity with respect to a user.

A robot, comprising:

a non-transitory computer readable medium configured to storeinstructions thereon;

a display in a face region of the robot; and

a processor connected to the non-transitory computer readable medium,wherein the processor is configured to execute the instructions for:

-   -   generating an eye image, wherein generating the eye image        comprises changing an apparent age of the eye image based on a        surrounding environment; and    -   instructing the display to display the eye image.

The robot according to the above combination, wherein the processor isconfigured to execute the instructions for:

extracting a feature quantity of an eye of a user from a captured imageof the user, and

changing the apparent age of the eye image based on the feature quantityof the eye of the user.

A robot, comprising:

a camera;

a non-transitory computer readable medium configured to storeinstructions thereon;

a display in a face region of the robot; and

a processor connected to the non-transitory computer readable medium,wherein the processor is configured to execute the instructions for:

-   -   detecting a line of sight of a user based on an image from the        camera;    -   generating an eye image, wherein the image from the camera is        superimposed on the eye image in response to the detected line        of sight being directed toward the robot;    -   instructing the display to display the eye image.

What is claimed is:
 1. A robot, comprising: a non-transitory computerreadable medium configured to store instructions thereon; a display in aface region of the robot; a processor connected to the non-transitorycomputer readable medium and to the display, wherein the processor isconfigured to execute the instructions for: extracting a featurequantity of an eye of a user from a captured image of a user; generatingan eye image, wherein generating the eye image comprises: setting anobservation period, selecting the user from the observation period, andgenerating the eye image to be reflective of the extracted featurequantity following elapse of the observation period; and instructing thedisplay to display the eye image.
 2. The robot according to claim 1,wherein the processor is configured to execute the instructions forcausing a size of a pupil image included in the eye image to change. 3.The robot according to claim 1, wherein the processor is configured toexecute the instructions for causing a form of an eyelid image, includedin the eye image, to change.
 4. The robot according to claim 1, whereinthe processor is configured to execute the instructions for causing anoperation pattern of the eye image to change.
 5. The robot according toclaim 1, further comprising: a detector configured to detect an externallight, wherein the processor is configured to execute the instructionsfor: causing a size of a pupil image, included in the eye image, tochange in accordance with an intensity of the detected external light,and causing a form of an eyelid image, included in the eye image, tochange in response to the intensity of the detected light exceeding athreshold.
 6. A robot, comprising: a non-transitory computer readablemedium configured to store instructions thereon; a display in a faceregion of the robot; a detector configured to detect an external lightsource; and a processor connected to the non-transitory computerreadable medium, wherein the processor is configured to execute theinstructions for: generating an eye image, wherein the eye imagecomprises a catch light, and generating the eye image comprises: settinga position of the catch light based on a direction of the detectedexternal light source relative to the robot, and setting a size of adistortion of the catch light based on the position of the catch light;and instructing the display to display the eye image.
 7. The robotaccording to claim 6, wherein the processor is configured to execute theinstructions for causing a size of the catch light to change inaccordance with a detected intensity of the detected external lightsource.
 8. A robot, comprising: a non-transitory computer readablemedium configured to store instructions and a plurality of iconsthereon; a display in a face region of the robot; and a processorconnected to the non-transitory computer readable medium, wherein theprocessor is configured to execute the instructions for: selecting anicon from the plurality of icons in response to satisfying of apredetermined condition; generating an eye image; superimposing theselected icon on the eye image in response to satisfying thepredetermined condition; and instructing the display to display the eyeimage and the superimposed icon.
 9. The robot according to claim 8,wherein the processor is configured to execute the instructions for:detecting a sight line of a user; and superimposing the selected icon onthe eye image in response to the detected sight line being directedtoward the robot.
 10. The robot according to claim 8, wherein theprocessor is configured to execute the instructions for generating theeye image comprising a catch light.
 11. The robot according to claim 10,wherein the processor is configured to execute the instructions forcausing a size of the catch light to change in accordance with adetected intensity of a detected external light source.
 12. The robotaccording to claim 10, wherein the processor is configured to executethe instructions for changing the catch light to the icon in response tosatisfying the predetermined condition.
 13. The robot according to claim8, wherein the processor is configured to execute the instructions forsuperimposing the icon on the eye image at a timing of executing ablinking of the eye image.
 14. The robot according to claim 8, whereinthe predetermined condition is an ambient environment condition.
 15. Therobot according to claim 8, wherein the predetermined condition is acharging rate of a battery of the robot.